In tele-immersive conferencing a subject is captured by a fixed set of cameras. This camera data then is transmitted to a remote site, and a viewer at the remote site sees the subject in real time from a moving virtual viewpoint. The view at this moving virtual viewpoint is reconstructed from the views of the fixed set of cameras. At any given time, various surfaces of the subject that are visible from the virtual viewpoint may not be visible to any of the cameras. This can make reconstruction of the view of those surfaces difficult.
It becomes desirable to reconstruct virtual views because at each site there are one or more depth color cameras capturing the scene, while remote users could view the scene from a different viewpoint. This reconstruction or restoration process is called dis-occlusion since the areas visible to the viewpoint of a remote user may be occluded to the cameras. Various techniques exist to reconstruct virtual views from a moving virtual viewpoint by filling dis-occluded areas. One class of techniques fits a captured frame into a model and then fills the dis-occluded areas. Another class of techniques uses multiple cameras to capture the scene from different positions. However, none of these techniques address the temporal aspect of virtual view reconstruction.